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ABSTRACT 

A system for evaluating the special services provided 
to disadvantaged and handicapped students enrolled in secondary-level 
vocational education programs in Vermont is presented. It was 
developed, pilot-tested and used to evaluate several local vocational 
special needs projects during the 1981-82 school year. The evaluation 
consists of three phases. The first phase is a self -evaluation 
conducted by local education agency (LEA) personnel and reported to 
the State. The second phase is an external evaluation, conducted by 
an evaluation specialist who spends 2'days at the respective LEA. The 
third phase is the development of a project improvement plan, based ^ 
on the outcomes of the evaluation activities. (Author/PN) 
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PREFACE 



The evaluation system, procedures, and materials presented in this 
document represent an approach used by the Vermont Division of Vocational 
Education to evaluate federal and state funded vocational special needs 
projects operated in local education agencies throughout the state. 
While this particular approach is designed to produce information needed 
by the State Department of Education, it also emphasizes the importance of 
local involvement- in the planning, administration and use of the evalif&tion 
for project improvement purposes. The collective input of a local 
evaluation steering committee in all phases of the evaluation and an in- 
depth project review session with an external evaluator are but two examples 
of the importance attached to local involvement in the evaluation effort. 

The procedures manual for local education agency pef^sonnel was 
developed during the 1981-82 school year. Following an extensive survey 
of evaluation systems and practices in several states , the Vermont system 
was conceived and presented in draft copy form for local and state education 
agency revtew. This prelimihary work was pi lot-Jtested at one vocational 
center and subsequently revised for use in evaluating special needs 
projects at another four vocational centers. Constructive feedback 
collected during these evaluations was helpful in preparing the present 
version of the procedures manual. 

Many individuals and agencies contributed to the development of this 
manual. The Director of the Vermont Division of Vocational Education, 
Arthur Ericson, was supp'ortive in securing the resources necessary to conduct 
the project. His critique of the materials at mid-point in the project 
was especially helpful. The State Consultant for Vocational Special Needs, 
Robert Watson, provided important feedback throughout each stage of the 
project. Bob's active interest and involvement added a valuable dimension 
to the project effort. We also commend him for assuming a leadership role 
in the ongoing operation ojf the vocational special needs evaluation system. 

Appreciation is extended to the personnel at the Rutland Area Vocational - 
Technical Center for their participation as a field-test site and for the 
valuable suggestions offered for improving the evaluation system. Special 
thanks is due to Lou Salebra, the vocational director; Denise Shannon, 
the vocational special needs instructor; and Les Johnson, the guidance 
coordinator at the Rutland Center. We also thank the many individuals from 
the four vocational centers who devoted much time and energy to the 
evaluation effort. 
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The authors are grateful for the support provided through the 
vocational education department at the University of Vermont. The 
clerical assistance provided by Janet Ferguson and Marsha Duprey was 
important to the successful operation of this project. 

Technical assistance funds provided by the Leadership Training 
Institute in Vocational and Special Education^t the University of 
Illinois were used for the initial printing /find duplication of this 
document. These funds, and the interest expressed in our project by 
Drs. James 'Green an and L. Allen Phelps, were much appreciated. We 
also thank Susan Hasazi , Director of the "Careers" Project at the 
University of Vermont, for her help in securing the Illinois funding. 

Although the printed version of the evaluation system may 
symbolize a completed product, we view it as a healthy beginning. 
Revisions will undoubtedly need to be made as the evaluation system 
matures and the needs of local and state education agencies shift. 
This formative view of the evaluation system is particularly important 
as we continually seek to provide a vocational education deli\^ry 
system that is responsive to the needs of disadvantaged and handeiapped 
youth. 

Leonard Albright, Project Director 
James Frasier, Pr©ject Coordinator 
August, 1982 
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INTRODUCTION 

Principally through support from federal vocational education 
legislation, the Vermont Division of Vocational-Technical Education 
has been able to assist local education agencies in establishing a 
wide variety of special services projects for disadvantaged and 
handicapped students. Without question, each special project has 
developed localized approaches for helping students with special 
needs succeed in vocational education programs. 

The evaluation system presented in this manual is intended to 
be responsive^ to the unique features and operational qualities of 
individual projects at the local level, while also fulfilling the 
information needs of the State Division. Therefore, this system 
attempts to obtain maximum input and involvement from local education 
agency (LEA) personnel in all stages of the evaluation process. 

The major purposes of the Vermont Division of Vocational -Technical 
Education's evaluation system for special needs projects are: 

1) to assist LEA staff in program and service improvement 
efforts ; 

2) to obtain information needed by the Division to determine 
individual program effectiveness, as mandated by the 1975 
Vocational Education Act Amendments (PL 94-482). 

The Evaluation System 

The evaluation framework presented in the remaining parts of this 

booklet consists of three, interrelated phases: 

I. Self-Evaluation 

II. External Evaluation 

III. lea's Program Improvement Plan 

A brief overview of each phase is offered on the next page. 



/. Self- Evaluation 

The purgpse of this phase i§ to ensure the full participation of 
the LEA in the development of information describing the operational 
qualities of its project. In the first segment of the self-eval uation, 
(Section A), a Local Evaluation Steering Committee prepares written 
descriptions of the project in response to a series of questions 
determined by the Division. The second segment of the self-evaluation. 
Sections B and C, essentially involves the Local Evaluation Steering 
Committee in collecting and analyzing^ information from the various 
groups associated with the project. ^ 



II. External ^valuation 

The purposes of this phase 
local project data reported tql 



are to; 1) verify the accuracy of the 
the Division, and 2) facilitate the 
development of project improvement efforts. This phase is accomplished 
by an external evaluator, who spends about ^h days at the respective LEA. 
During the first day, the evaluator reviews local data and interviews 
individuals familiar with the project. On the second day, the evaluator 
conducts, an indepth review of the proj<ect with the full membership of 
the Local Evaluation Steering Committee. This review se^ssion is intended 
to aid the LEA in identifying initial recommendations for project 
improvement (Phase III). 

i 

ni. LEA Project Improvement Plan * - 

The purpose of this third and final phase of the evaluation system 

is for the LEA to utilize information gained from the self-evaluation 

and external evaluation activities to chart a plan of improvement for 

the local special needs project. The writing of this plan is done at 

the local level and presented to the State Consultant for Special Needs 
for review and approval. 
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The Orientation Session 

In order to get the evaluation off to a good start, the State 
Consultant for Vocational Special Needs conducts an orientation session 
with key representatives from the local education' agency. Commitments 
from these representatives to serve a3 members of the Local Evaluation 
Steering Committee should be obtained prior to the orientation session 
with the State Consultant. The purpose, composition and major functions 
of this steering committee are highlighted on the next page. 

At the orientation session, the State Consultant explains the 
purposes of the evaluationj describes the central phases of the system 
and presents the materials that need to be organized and submitted to 
the State Di 'ision of Vocational Education. A calendar of major 
evaluation activities, with corresponding timelines, is also determined 
during the orientation meeting. A copy of the calendar to be finalized 
at this session appears on page £ of this introductory section. 



GUIDEL INES FOR THE 

. ^ . 

LOCAL EVALUATION STEERING COMMITTEE 



The purpose of the Local Evaluation Steering Committee 

is to act as a vehicle through which all matters concerning 

this evaluation effort can be channeled. 

To provide for adequate representation, a minimum membership 
of the Local Evaluaition Steering Committee must include, 
but need not be limited to, the following individuals: 

Vocational Center Director 

Special Needs Project Instructt)r 

Vocational Faculty Member 

Other members may be added as determined appropriate by 
the' local education agency (e.g. representation from 
stijdent, parent, guidance, and advisory committee groups). ' 

The functions of the Local Evaluation Steering Committee 
members include the following: 

1. Attend an orientation meeting with the State 
Consultant for Vocational Special Needs. 

2. Establish an evaluation calendar with the State 
Consultant. 

3. Participate in the development, review, and approval 
of information submitted to the State Consultant, 

as requested in PHASE I - Section A of the evaluation. 

4. Determine the most effective procedure(s) to use in 
obtaining faculty, student and administrator responses 
to the written questionnai res (e.g. direct interview, 
mailed survey). 

5. Establish who will be res|ionsible for data analysis 
and reporting, as described in PHASE I - Section C. 

6. Meet with the external evaluator for indepth discussion 
of the project (PHASE II). 

7. Assist in the preparation of a local plan for project 
improvement (PHASE III)i 



EVALUATION CALENDAR 



The dates for the evaluation activities described below are to be 
established by the Local Evaluation Steering Committee and the State 
Consultant at the initial orientation meeting. 

DATE 

_• 1. First meeting of Local Evaluation Steering Committee to 

consider information for inclusion in Section A of the 
self-evaluation (PHASE I). This meeting should be held 
within _5 working days following the orientation session 
with the State Consultant. 

2. Committee meeting to approve written version of Section A 

for s-ubmission to the State Consultant. 

3. Send Section A to State Consultant. This is to be done 

by no later than 10 working days after the orientation 
meeting with the State Consultant. 

4. Date by which State Consultant will send prepared 

questionnaires to Steering Committee. This is to be done 
within 7^ working days after Section A has been received 
by the State Consultant. 

5. Date Steering Committee will have collected required 

percentages of questionnaires as described in Section B 
of PHASE I. This is to be done within JO working days 
after the prepared questionnaires have been received 
from the State Consultant. 

6. Date by which analysis of questionnaire data (Section C - 

PHASE I) will have been completed by the Steering 
Committee. This is. to be done by no later than 2 working 
days after the date agreed upon in #5 above. 

7. Dates of External Evaluator two-day visit (PHASE II). 



8. Projected return date of external ^valuator's report by 
State Consultant to the LEA. ^ 

9. Target date for Steering Committee meeting to review 
external evaluator' s report and prepare draft of Local 
Project Improvement Plan (PHASE III). 

10. Target date for State Consultant's visit to LEA for 

review and discussion of Local Project Improvement Plan. 
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PHASE I: SELF-EVALUATION 



Instruments and Procedures 
for 

Local Evaluation Steering Committee 



7 



14 



1 




SECTION A 



PROJECT DESCRIPTION 



PURPOSE The purpose of Section A is to secure 

accurate information about your center's 
special needs project. This information 
will be used in developing questionnaires 
for vocational faculty ^ project staff and 
administrative personnel and reviewed by 
the external evaluator. 

* 

INSTRUCTIONS After the Evaluation Report cover page 

and Section A have been completed and 
typed, please submit this material to: 

Special Needs Consultant 

Division of Vocational-Technical Education 

Vermont Department of Education 

State Office Building 

Montpelier, Vermont 05602 

RECOMMENDATIONS In preparing this section, the Steering 

Committee should make use of the following 
documents: the original funding proposals, 
subsequent funding request documents, and 
annual project reports. 

Since information provided in this section 
will be utilized to develop questionnaires 
and reviewed by the external evaluator, it is 
recommended that ALL members of the Steering 
Committee carefully examine this important 
information before submitting it to the 
State Consultant for Special Needs. 
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Date 



SPECIAL NEEDS VOCATIONAL PROJECT 
EVALUATION REPORT 
for 
Section A 



(Project Title) 
(School District) 
(Phone Number) 



LOCAL EVALUATION STEERING COMMITTEE 
MEMBERSHIP 

Name Position/Title 

Vocational Center Director 



Special Needs Project Instructor 
Vocational Faculty Member 



/ 



*Please place an asterisk (*) next to the name of the individual who will 
serve as the contact person for this evaluation. 



NARRATIVE DESCRIPTION OF PROJECT 



On this page, please type a brief narrative description of your project, 
which includes its purpose(s) and special services provided to students. 
Please keep your description within 100 words. 



i 
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PROJECT OPERATIONAL GOALS AND ENABLING OBJECTIVES 



On this page, please type each major project goal with its enabling 
objecti-ves. 
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STUDENT IDENTIFICATION AND REFERRAL 



Describe 1n the space below, the criteria used to determine 
student eligibility for your project. 



\ 



\ 



Describe in the space below, the procedures used to identify students 
in need of your project's services. 



is 
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STUDENT SELECTION CRITERIA 



Describe, in the space below, the criteria used to select students for 
participation in your project. 



✓ 
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INDIVIDUALIZED PLANNING PROCEDURES 



Describe, in the space below, the procedures used in planning for 
the individual needs of students enrolled in your project. - 



Identify, in the space below, the persons ^who are involved in planning 
for the individual nee;Ets of the students enrolled in your project. 
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MONITORING STUDENT/PROJECT ACHIEVEMENT 



1 



Please describe, in the space below, what provisions are made for monitor- 
ing the performance of students enrolled in your project. Also, describe 
who receives this information, when and how it is used. 



Please describe, in the space below, the procedures , used by the project's 
staff to periodically check on the performance of the project. 
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STUDENT TERMINATION/FOLLOW UP 



Please describe, in the space below, the procedures used to determine 
when a student is no longer in need of receiving further services 
from your project. 



Please describe, in the space below, the' procedures used for the follow- 
up of students who are no longer receiving special services from your 
project, but who are presently enrolled in a vocational education program. 



V 
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EXAMPLES OF PROJECT SUCCESS 



On this page, provide at least 5 examples which strongly suggest that 
the project has enabled individual students to succeed in their regular 
vocational education programs. Please be brief and to the point in your 
descriptions. 
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UTILIZATION 'OF SCHOOL AND COMMUNITY RESOURCES 



Please describe, in the space below, efforts made by your project to 
utilize resources and services available for special needs students 
within th^ school districts and communities' that have students enrolled 
at the center^ (e.g. senior citizen volunteers, big brother/big sister 
programs, regional mental health services, peer tutors, advisory 
committee members who are handicapped/disadvantaged) . 
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SPECIAL NEEDS S'TUDENT PARTICIPATION RATES FOR LAST YEAR 

Male Female Disadvantaged Handicapped Total 

Total number of special needs 
students enrolled at the 
vocational center last school 

year 

Number of special needs 
students enrolleci at the 
vocational center last school 
year who were enrolled 

irTtWs project __ 

Total number of special needs 
students enrolled at the voca- 
tional center last school 
year who DROPPED OUT of the 

vocational center ^ 

Number of special needs 
students enrolled at the 
vocational center last school 
year who were enrolled in 
this project and who 
DROPPED OUT of the vocational 

center 

Total numb.er of special 
needs students enrolled at 
the vocational center last 
school year who DROPPED OUT 
OF SCHOOL COMPLETELY 



Number of special needs 

students enrolled at the 

vocational center who were ^ 

enrolled in the project and ^ 

who DROPPED OUT OF SCHOOL 

COMPLETELY 



COMMENTS CONCERNING THE ABOVE FIGURES BY THE LOCAL EVALUATION STEERING COMMITTEE: 
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SPECIAL NEEDS STUDENT PARTICIPATION RATES FOR CURRENT SCHOOL YEAR 



Male Female Handicapped Disadvantaged Total 

Total number of special needs 
students enrolled at the voca- 
tional center this school year. " 

Number of students at the 
vocational center this school 
year who are enrolled in 

the project 



Total number of special needs 
students enrolled at the voca- 
tional center this school year 
who have DROPPED OUT of the 
vocatT onal center 



Number of students at the 
vocational center this school 
year who were enrolled in this 
project but have DROPPED OUT 
of the vocational center 



Total number of special needs 
students enrolled at the voca- 
tional center this school year 
who have DROPPED OUt OF 
SCHOOL COMPLETELY 



Number of students enrolled 
at the vocational center this 
school year who were enrolled 
in this project and who 
DROPPED OUT OF SCHOOL 
COMPLETELY 



Number of vocational students 
receiving project services 
this year who also received 
project services last year.. 



Number of vocational students 
not receiving project services 

this year who did receive ' 
project services last^year.. 

COMMENTS CONCERNING THE ABOVE FIGURES BY THE LOCAL EVALUATION STEERING COMMITTEE: 
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STUDENT ENROLLMENT IN SPECIAL NEEDS PROJECT 

BY 

VOCATIONAL PROGRAM AREA FOR CURRENT SCHOOL YEAR 

Total Enrollment Project Enrollment No. of Project 

Name of by Voc. Program by Voc. Program Students Served by 

Vocational Program Grade Level 

Hand. Disad. Hand. Disad. 9th 10th 11th 12th 



9 

. TOTALS 



Total Number of D.O. students in this special needs project. 
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SECTION B 
INTERNAL REVIEW INSTRUMENTS 



PURPOSE 



INSTRUCTIONS 



This section of the evaluation has been developed to help 
"^assess student, faculty, and administrator understandings of 
your project's operation- 

Information submitted by the Local Evaluation Steering Committee 
in Section A will be used to complete the "box". areas of the 
questionnaire format shown in this section- The typing and 
reproduction of the questionnaire format will be done by the 
State Division of Vocational Education and^sent to the Local 
Steering Committee for distribution to the individuals to be 
surveyed- The typed questionnaires returned by the State 
Division are to be completed by vocational center faculty, 
project, staff and administrators- A separate student question- 
naire appears after the faculty, project staff and administrator 
questionnaire- 

A sample covering letter to be attached with each questionnaire 
appears on the next page- 

The groups and individuals that must be surveyed and the 
respective minimum returJl Yates are listed below: 

• At least an 80% return rate from all vocational faculty 
(that is, faculty who presently are using the project's 
services as well as those who are not)- 

• At least an 80% return rate from all students who have 
received project services during the school year- 



t AV] special needs project staff. 

• The Vocational Center Director- 

• The Vocational Center Guidance Coordinator. 

RECOMMENDATIONS While each questionnaire format contains several questions 
or areas that must be addressed, the Local Evaluation 
Steering Committee is encouraged to add other questions that 
are also felt to be important in determining local program 
needs. ' ^ 



Also, in order to obtain a more comprehensive picture of . \ 
the project, the LEA is encouraged to survey other groups 
affected by this program (e.g. sending school special educators, 
advisory committee members, parents). 
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SAMPLE MEMORANDUM FROM THE LOCAL EVALUATION . 
STEERING COMMITTEE TO INDIVIDUALS RECEIVING QUESTIONNAIRES ^ 



TO: ' Personalize (e.g. vocational center faculty) 

FROM: , Vocational Director 

, Special Needs Staff Member 

, Faculty Representative 

SUBJECT: Special Needs Questionnaire 



We are in the process of evaluating the ( ) Project. Since we 

feel that you are an important .part of this project, your feedback 
on services and project effectiveness is very necessary to this 
evaluation. 

Please complete the questionnaire and return it by ( date ) to the 
secretary in the vocational office. 

Thank you for your assistance in this very important activity. 



Adapted from Wentling, T.L. Evaluating Occupational Education 
and Training Programs , Second" Edition. Boston: Allyn and Bacon, 
Inc. , 1980. 



.30 
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SPECIAL NEEDS PROJECT 
EVALUATION QUESTIONNAIRE 



Present Position: ( 

VocatioVal Teacher Guidance 

^Special Needs Instructor S ending School Special Educator 

Vocational Administrator Other (Please list position) 



If you are a vocational teacher, please place a check next to the appro- 
priate statement, 

^I- presently have or have had students enrolled in this project. 

I have not had students enrolled in this project. 
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In the box below is a brief narrative description of your center's 
project for special needs students. Please read this description and 
respond to each question below by circling the appropriate number. 



1. Does the above description accurately reflect your understanding of this 
project? ^ 



1 

Does not reflect 
my understanding 



Uncertain 



Very accurately 
reflects my 
understanding 



2. Has this project been successful in helping special needs students 
succeed in their regular vocational program? 



Verv successful 



Uncertain 



Not successful 



3. Is this project needed to help* special needs students succeed in their 
regular- vocational program? 

1 2 3 4 5 

Not really Uncertain Very definitely 

Please explain your answer to question #3 
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In the box below is a description that explains the criteria used 
to select students for participation in your center's project for 
special needs students. Please read the description and respond 
to each question below by circling the appropriate number. 



4. Does the above description accurately reflect your understanding of the 
criteria used to select students for. participation in this project? 



1 

Does not reflect 
my understanding 



Uncertain 



Completely 
reflects my 
understanding 



5y Are these selection criteria appropriate for enrolling those students 



most in need of this project's services? 



1 



Not appropriate 



Uncertain 



Very appropriate 



6. Do students in this project meet the above stated seTe^ion criteria? 
,1 2.3 4 5 



V 
' No 



UrTcertain 



Very definitely 
PLEASE EXPLAIN MORE FULLY YOUR ANSWER TO #6 
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In the box below is a description that explains the individual 
planning that takes place with students enrolled in your center's 
project for special needs students. Please read this description and 
respond to each question below by circling the appropriate number. 




7. Does the above description reflect your understanding of the individual 
planning that takes place for students enrolled in this project? 

5 4 3 2 1 

Completely 

reflects my Does not reflect^ 

understanding Uncertain my understanding 

8. . Have these procedures been effective in planning individualized services 

for students enrolled in the project? 

1 2 3 4 5 

Not effective Uncertain Very effective 



9. Have you ever been directly involved in the planning of special services 
. for students enrolled in .this project? 

\ 1 Z 4 . 5 

I have never I am I am invblved in 

been involved occasionally involved planning regularly 

WILL YOU PLEASE EXPLAIN YOUR ANSWER TO #9 
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In the box below is a description that explains the procedures used 
for monitoring the performance of students enrolled in your center's 
project for special needs students. Please read the description and 
respond to each question below by circling the appropriate number. 




10. Does the above description reflect your understanding of how student 
performance is monitored in this project? 


1 2 3 4 




Does not reflect 
' my understanding Uncertain 


^^Very accurately . 
reflects my 
understanding 


11. Do you think the above procedures are adequate 
progress? ^ ^ 


for monitoring student ^ 


12 3 - 4 
Not adequate Uncertain 


5 

Very adequate 


12. When one of your students is enrolled in this project, how often are you 
involved in discussions with the staff concerning his/her progress? 


5 . 4 3 2 

I am involved . I am 
regularly in the occasionally 
monitoring of progress ' involved 


1 

I have never 
been involved 


PLEASE EXPLAIN YOUR ANSWER TO #12 
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Listed below are the major goals of this project. Please review 
these goals and respond to each question below by circling the 
appropriate number. 



13. Do the above statements accurately reflect your understanding of the 
project's goals? 



1 2 3 4 5 

Does not reflect Uncertain The list very 

my understanding . accurately reflects 

of the project's my understanding of 

goals the project's goals 

14. Has the project been able to implement: 

Has been Has not been 

implemented I don't know implemented 

GOAL #1 3 2 1 

GOAL #2 3 2 ' 1- 

GOAL #3 ,...3 2 1 

GOAL m 3 2 1 • 

15. How well do you think the project is currently implementing: 

Not well I don't know Exceptionally 

well 

GOAL #1 ...........1 . ' 2 3 

GOAL #2 1 . 2 ■ 3 ' , 

GOAL n ^ . . . 1 ' .2 3 ■ 

GOAL M 1 2 3 
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16. Do you think students who receive services from this special project 
would be able to succeed in their regular vocational program without 
the exi stance of this^ project? 

^ 1 / ^ 2 3 

No, most I really Yes, most 

would fail don't know would pass 

without it 

17. Do you regard this project as making a critical difference in the 
special needs student being able to pass his/her vocational program? 

3 2 1 

Yes Uncertain No 

If YES: Would you please provide a specific example of how 
project has made this difference? 



18. Based on your knowledge and experience with this project, what 
suggestions would you offer for strengthening it? 
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STUDENT QUESTIONNAIRE 

1. How did you find out about this (title of project)? 

2. What -kind of help do you get from this project? 

3. How long. have you been coming to this project for help? 

4. How many days each school week do you come to this project? 

5. How much time do you spend in this project each time you come? 

6. How much has this project helped you in your vocational class? 

1 2 3 4 5 

I would make A little Some . A lot I couldn't make 

it without ) it without this 

this help \ help 

7. Who will decide when you don't need this project's help any more? 

8. What do you work on in this project that helps you to do well in your 
vocational class? 

9. What do you like most about this project? 

10. What do you not like about this project? 

11. Do you help plan what you will do in this project? 

Yes How? . . 

^ No, my vocati onaij^teacher does 

No,' this project's iXcher does 
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SECTION C 

PROFILE OF FACULTY AND STAFF RESPONSES 



PURPOSE The purpose of this section is to organize the 

questionnaire results for use in discussions with 
the Local^ Evaluation Steering Committee and the 
external evaluator during the second day of the 
evaluator*s visit. 



LNSTRiJCTlONS Copies of the tabulated results are to be made for 
each member of the Steering Committee and the 
externa] evaluator. 

RECOMMENDATIONS When calculating and recording the mean scores in 

Section C, it is suggested that the Steering Commit- 
tee work in groups of two persons - one person 
reading the circled numbers as the other person 

I records and calculates the mean score. 

It is further suggested that one person from the 
Local Evaluation Steering Cormiittee assume the 
responsibility for recording faculty and staff 
"comments to selected questions in Section C. 
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CUMULATIVE RATINGS OF FACULTY AND STAFF 



CO 



Take each question listed below (and on the following pages) and calculate the mean score of each 
group questioned. Use the following method for calculating the mean score : 

STEP #1 - Total all circled numbers of a group's responses for one question. 

Then, divide this total by the total number of a group's respondents . 

EXAMPLE : (54) sum of all values circled for question #1 by faculty 
r\2) total number of faculty respondents 



or 



54 
12 



4.5 for the faculty mean score 



STEP §2 - Take the calculated mean for each group and record the number to the nearest 
tenth under the appropriate group of respondents. 



QUESTION 
Project Description 

1. Does the above description accurately reflect your understanding of 
this project? 

2. Has this project been successful in helping special needs 
students succeed in their regular vocational programs? 

3. Is this project needed to help special needs students succeed 

in their regular vocational program? 

Selection Criteria 

4. Does the above description accurately reflect your understanding 
of the criteria used to select students for participation in 

this project? 



FACULTY 



PROGRAM 
STAFF ADMIN. 



GUIDANCE 
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QUESTION 

5. Are these selection criteria appropriate for enrolling 

* * those' students most in need of this project* s services?.... 

6. Do students in this project meet the abUve stated 
selection criteria? 

Individual Planning ' , 

7. Does the above description reflect your understanding of the 
individual planning that takes place for students enrolled 
ia^this project?. 

8. Have the procedures been effective in planning individualized 
services for students enxolled in the project? 

^ 9. Have you ever been directly involved in the planning of 
^ special services for students enrolled in this project?.. 

ilonitoring Procedures 

10. Does the above description reflect ^your understanding of 
how student performance is monitored in this project?... 

11. Do you think the above procedures are adequate for 
monitoring student progress?... 

12. When one of your students is enrolled in this project, how 
often are you involved in discussions with the staff 
concerning the student's progress?......... 
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Project Goals 

13. Do the above statements accurately reflect your understanding 
of the project's goals? ^. 

14. Has the project been able to implement: Goal #1 

Goal #2 



^ A r 
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FACULTY STAFF ADMIN. GUIDANCE 
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CUM.ULMlVi_AATJNGS OF ,FACyLlY_ _ANp_iJAFF 



PROGRAM 

QUESTION FACULTY STAFF ADMIN. GUIDANCE 

i4. Continued - Has the project been able Goal #3 • _____ " r 

to implement: 

Goal #4 



15. How well do you think the project is currently 

implementing? Goal #1 

Goal #2 



Goal #3 



Goal $4 



Other 

16. Do you think students who receive services from this special 
project would be able to succeed in their regular vocational 
program without the exi stance of this project? 

17. Do you regard this project as making a critical difference 
in the special needs student being able to pass his/her 
vocational program? 
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PROFILE OF FACULTY AND STAFF COMMENTS 



On this page - and additional pages if necessary - please type all comments made by 
individuals within each arjQup to ausstion J3 of the questlonaaire : 



Question #3 - - Is this project needed to help special needs 

students succeed in their regular vocational program? 

\ 



VOCATIONAL FACULTY COMMENTS 



PROJECT STAFF COMMENTS 



ADMINISTRATOR COMMENTS 



GUIDANCE PERSONNEL COMMENTS 
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^ PROFILE OF FACULTY AND STAFF COMMENTS 



On this page - and additional pages if necessary - please type ALL comments 
made by individuals^ within each group to question #6 of the questionnaire. 

I 

I : — 



Question #6 - - Do students in this project meet the 
above stated selection criteria? 



VOCATIONAL FACULTY COMMENT^^ 


PROJECT STAFF COMMENTS 

• 


ADMINISTRATOR COMMENTS 


GUIDANCE PERSONNEL COMMENTS 


















■ 

• 
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PROFILE OF FACULTY AND STAFF COMMENTS 



On this page - and addition pages if necessary - please type ALL comments made 
by individuals within each group to question #9 of the questionnaire. 



Question #9 - - Have you ever been directly involved in the 
planning of special services for students 
enrolled in this project? 



VOCATIONAL FACULTY COMMENTS 



PROJECT STAFF COMMENTS 



ADMINISTRATOR COMMENTS 



GUIDANCE PERSONNEL COMMENTS 



PROFILE OF FACULTY AND STAFF RESPONSES 



Onr thrs page - anrf additional pages if necessary - please type ALL 
comments of the vocational faculty to question #12 of the 
questionnaire. 



Question #12 - - When one of your students is enrolled 
in this projects how often are you in- 
volved in discussions with the staff 
concerning his/her progress? 
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PROFILE OF FACULTY AND STAFF COMMENTS 



On this page - and additional pages if necessary - please type ALL comments made by 
individuals within each group below to question ^16 of the questionnaire. 

Question ^17- - Would you give a specific example of how this project has 

made a critical difference 1n a special needs student being 
able to pass his/her vocational program? 



VOCATIONAL FACULTY COMMENTS 


PROJECT STAFF COMMENTS 


ADMINISTRATOR COMMENTS 


GUIDANCE PERSONNEL COMMENTS 











I 

.1 
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PROFILE OF FACULTY AND STAFF COMMENTS 



On this page - and additional pages if necessary - please type ALL comme/its made by 
individuals within each group below to question #18 of the questionnaire. 

^ f 

Question #18 - - Based on your knowledge and experience with thi^ project, 
what suggestions can you offer for strengthening it? 



VOCATIONAL FACULTY COMMENTS 



PROJECT STAFF COMMENTS 



ADMINISTRATOR 



COMMENTS 



GUIDANCE PERSONNEL COMMENTS 
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PHASE II: EXTERNAL EVALUATION 



EvatuatoT Activity Schedule and 
Committee Member Responsibilities 
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EXTERNAL EVALUATION 

Information in this section is to help the local steering comnittee 
prepare for the "external evaluation. This phase of the special needs 
project evaluation consists of a ^h day visitation by an evaluation 
specialist from outside the school district. The specialist is selected 
and trained by the State Division of Vocational Education. 

During the 1^ day period, the evaluator analyzes the results from 
the self-evaluation questionnaires; interviews selected faculty members, 
administrators and students; Wid m eets with the local evaluation steering 
committee to discuss her/his observations. Following the review and 
discussion session with the local steering committee, the evaluator prepares 
a project evaluation report for the State Division of Vocational Education. 
Upon acceptance by the State Consultant for Special Needs, the report is 
used by the State Consultant and the local LEA in negotiating a local plan 
for strengthening the special needs project* 
Steering Committee Responsibilities 

The three basic responsibilities of the local steering comnittee during 
the external evaluation are: 

1. Helping the evaluator finalize his/her schedule 
of activities for the ]k day review. 

2. Being accessible during the Ik day time period. 

3. Participating in a review and discussion session 
during the second day of the evaluation. 

Each responsibility is djscussed on the following page. 
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Finalize Ori-.^itc .'Schedule 



An outline of the evaluator's "on-site" schedule 
appears on the following page. As seen in this out- 
line, the first activity is a meeting with the 
evaluator to complete the scheduling. At this time, 
the individuals to be interviewed by the evaluator 
and the timeline^ for each activity are to be deter- 
mined. A completed copy of Sections A (Project 
Description) and C (Profile of Faculty and Staff 
Responses) is to be given to the evaluator and to 
each committee member at this initial meeting. 

■ : . I< * : i n j A ? jc 3 s i I - /. * : 
\ 

Inj addition to scheduled meetings with the evaluator 
during the V2 day review, the Steering Committee 
members should also be available for individual 
consultations. Try to avoid scheduling any activi- 
ties that could interfere with the evaluator' s 
visitation. 

On the morning of the second day of the evaluation, 
the evaluator meets with the steering committee to 
share observations and discuss the project operation. 
Since this "wrap-up" meeting is when the committee 
and the evaluator get into the nitty-gritty of the 
project by discussing project strengths and weaknesses, 
it is critical that each member of the steering committee 
attend this important session. Each participant should 
prepare for this session by reviewing the information 
reported in Sections A and C. Worksheets for use by 
the committee members during the review and 
discussion session with the evaluator are located at 
the end of this section. 

The active and full participation of all Local Evaluation Steering 

Committee members in the sessions with the external evaluator is essenfial 

for ensuring a meaningful and useful project evaluation. By making sure 

that the important aspects of the special needs project have been examined 

and that the recommendations for strengthening the project are accurate 

and realistic, the committee's contribution to improving the quality of 

vocational programs and services for special needs, students at the center 

should be substantial. 
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SCHEDULE FOR EXTERNAL EVALUATOR^S VISIT 



TIME FIRST DAY*S ACTIVITIES 

8:00 - 8:30 AM Meeting with Local Evaluation Steering Committee members 

to agree upon schedule for the evaluation and make all 
necessary arrangements. 

8:30 - 11:30 AM External evaluator's verification^ of returned ques- 

tionnaire percentages, checking of response record- 
ings, initial review of local self-evaluation / 
documents 

Lunch 

Meeting with LEA special needs project staff 

Interview one student not presently receiving 
project services, but has received such services 
in the past 

Interview one student currently using the project's 
services 

Meeting with a vocational faculty member who 
presently has a student enrolled in the project 

Meeting with a vocational faculty member who 
previously had a student receiving project services 

Meeting with the Center Director 

Dinner 

^ Evening woflc-up of project information for meeting 

with the Local Evaluation Steering Committee 

SECOND DAY'S ACTIVITIES 

8:15 - 11:00 AM . Meeting with alj[ members of the Local Evaluation 

Steering Committee 

11:00 - 12:00 AM Preparation of "exit report" 

12:00 - 1:00 PM Lunch with special needs project staff 
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RECORDING FORM 
FOR 

REVIEW SESSION WITH EXTERNAL EVALUATOR 



In the wrap-up session with the external evaluator, project strengths 
and weaknesses will be discussed along with recommendations for improve- 
ment. The form that follows is for your convenience in recording 
key information discussed during this session. 
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Date: 



SPECIAL NEEDS PROJECT EVALUATION 
Review Session with External Evaluator 



I. Areas of Project Strength 



II. Project Areas in Need of Strengttjening : 
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A [ Recording Form Continued 



III. Recommendations for Improving the Project; 



6\i 
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PHASE III: SPECIAL NEEDS PROJECT IMPROVEMENT PLAN 



Guidelines and Example Format 
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PROJECT IMPROVEMENT PLAN 



Following the external evaluation and receipt of the evaluation 
report, the LEA is to prepare a plan of improvement for the special 
needs project. This plan is to be presented to the State Special Needs 
Consultant for review and approval. 

The individuals responsible for preparing the local project 
improvement plan would most logically come from the membership of the 
Local Evaluation Steering Committee. However, since the project is a 
centerwide effort, other key individuals (e.g. sending school personnel, 
advisory members) could become involved in preparing the plan. This 
strategy would be particularly important if the improvement areas are 
directed toward these individuals/groups. 

Two documents should be used in preparing the written local 
improvement plan: 1) the recording forms used in the review meeting with 
the external evaluator, and 2) the external evaluator's written report. 

A sample format for preparing a local improvement plan appears on 
the following pages. 
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PLAN OF IMPROVEMENT 
' FOR 

(Project Title) 
(School Districtl 



PREPARED 
BY 



(Name) (Title) 



Date 



"^Person to contact for further information 
about this plan. 
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IMPROVEMENT 

AREAS 



PROCEDURES FOR 
ACCOMPLISHING IMPROVEMENT 
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STARTING 


ANTICIPATED 


PERSONNEL 


DATE 


' COMPLETION DATE 


RESPONSIBLE 









